Skip to content

Trim the encoder delay and padding of AAC ADTS streams - #157

Open
chrizzchrizz wants to merge 5 commits into
dimitris-c:mainfrom
chrizzchrizz:fix/aac-adts-assumed-delay
Open

chrizzchrizz wants to merge 5 commits into
dimitris-c:mainfrom
chrizzchrizz:fix/aac-adts-assumed-delay

Conversation

@chrizzchrizz

Copy link
Copy Markdown
Contributor

Trim the encoder delay and padding of AAC ADTS streams

Based on #150 (which is based on #144). Until those are merged this PR also shows their commits; the new change is the last commit only.

Problem

AAC-LC in an ADTS stream cannot carry gapless info: there is no packet table and no iTunSMPB. This is the format servers usually stream when they transcode to AAC live. Navidrome, for example, uses ffmpeg -c:a aac -f adts and serves it as audio/aac with no Content-Length. The decoder then plays the encoder delay at the start and the padding at the end as audio. Between two entries of a continuous mix this gives a gap of about 1640 samples (1024 priming plus the end padding), which you hear as a tick.

Changes

This applies the approach of #150 for MP3 without Xing/Info header to ADTS:

  • When an AAC-LC stream in an ADTS file (kAudioFileAAC_ADTSType) reports no packet table info, assume the encoder delay of ffmpeg's native AAC encoder (1024 frames). AAC in MP4 and HE-AAC are not affected.
  • At the end of the stream, drop the silent padding, as for MP3, but allow up to 2048 frames for AAC (the rest of the last frame plus one flush frame) instead of 1728 for MP3.
  • assumedPrimingFrames now also takes the file type, which is read from kAudioFileStreamProperty_FileFormat.

Testing

  • New unit tests in GaplessTrimTests: priming is assumed for AAC in ADTS, but not for AAC in MP4, HE-AAC in ADTS or other formats. The maximum padding is checked per format.
  • swift test: all tests pass.
  • Measured with two consecutive tracks transcoded to AAC ADTS by Navidrome (ffmpeg) and compared sample by sample with the source. The gap at the transition went from 1640 samples to 2 samples, for the live stream and for a downloaded .aac file. That matches the MP3 result of Trim the encoder delay and padding of MP3 streams without gapless info #150.

Decoders output the encoder delay (priming) and padding (remainder) of
formats like AAC and MP3 as audio. The packet table info was only used
for the duration, so consecutive entries were separated by a short
silence (2112 + remainder frames, about 50 ms, for AAC).

- Keep only the decoded frames in [priming, priming + valid) while the
  decoded position is exact (decoding from the start of the stream)
- After a seek, drop the remainder at the end of the stream instead
- Frame counts are converted to the output sample rate
- Mark the decoded position as inexact when re-reading from a seek offset
  (reset(decodedPositionExact:)), so mid-stream data isn't trimmed as the start
- Fall back to trailing-only trimming when packet table info arrives late
- Account for frames after a fully filled decode so they aren't overwritten
- Pass the EOF entry to trimTrailingGaplessPadding instead of looking it up
- Extract the trim math into GaplessTrim and add unit tests
- Keep the decoded position exact when seeking to the start of the data
- Ignore repeated packet table info once the gapless values are set
- Round the end of the kept range once to avoid an off-by-one with resampling
An MP3 without Xing/Info header reports no packet table info: a transcode of a server (live or from its cache), or a file of a simple encoder. The decoder then outputs the encoder delay at the start and the padding at the end as audio, so two entries are separated by a gap of about 35 ms, which is a tick in a continuous mix. The trimming added for AAC and tagged MP3 does not apply.

- Assume the encoder delay of LAME (576 frames, the decoder delay is already compensated by AudioToolbox) for an MP3 that reports no packet table info when it is ready to produce packets. The packet table info of a file with header is reported before and takes precedence.
- The padding is unknown. At the end of the stream drop the frames at the end of the renderer buffer that are below 20 percent of the peak level of the music before them (at most one frame plus the encoder delay, and only frames that are not played yet). The codec rings for a few dozen frames after music that ends abruptly, so a fixed silence threshold is not enough.

Measured with the end of one track and the start of the next one of a DJ mix (M4A transcoded to MP3 by Navidrome, with and without its transcoding cache), played back to back and compared sample by sample with the lossless source: gap at the transition 1535 frames (cached) / 659 frames (live) -> 0 frames, on macOS and on an iPhone 15 Pro. A sine cut in two parts, MP3 files with and without Xing header, and RAW: no gap, no phase jump.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant